\chapter{Agile Development Process}

Software development is a complex effort. Especially as the software industry evolves towards the direction of the enterprise-level application, the fact makes the complexity impossible to be ignored in the software development. The software development process is completely intellectual, and all of its intermediate products are marginal representations of the thoughts involved. The materials used to create the end product are extremely volatile: user requirements for a program the users have to see, the interoperation of other programs' signals with the program in question, and the interaction of people which are the most complex entities on the planet.

Besides, the feedback loop between customer and developer, between wish list and implementation, between investment and return on investment can be fairly long. Again, complexity plays a role here. When a system is simple, it is not so hard to know in advance what to do. But when the project is developed by hundreds of people all over the world and with the technology that that will not stand still, learning through short cycles of discovery is the tried-and-true problem-solving approach.

\section{Overview and Definitions}

Agile software development is a conceptual framework for software engineering that promotes development iterations throughout the life-cycle of the project. There are many agile development methods. They are attempting to offer once again an answer to the eager business community asking for lighter weight along with faster software development processes. This is especially the case with the rapidly growing software industry as well as for the emerging mobile application environment.

As a group of software practitioners and consultants in 2001 published the \textit{Agile Software Development Manifesto}\footnote{agilemanifesto.org and www.agilealliance.org, (1.5.2002)}, the focal values honored by the agilists are presented in the following:

\begin{itemize}
	\item Individuals and interactions over processes and tools
	\item Working software over comprehensive documentation
	\item Customer collaboration over contract negotiation
	\item Responding to change over following a plan
\end{itemize}

The key values that the Agile community respects are:

\begin{description}
	\item [First] The Agile principle emphasizes the teamwork and relationship of software developers and the roles specified in the contracts, rather than the dogmatic theories and processes. In the running agile projects, it is reflected in tight team relationships, close working environment arrangements and other means motivating team members.
	\item [Second] The important aim of the software team is to keep working out the running software. New versions are released on a regular and frequent basis. In some extreme approaches, new releases can come out hourly or daily. But usually they are produced every month or every two months. Therefore, the developers are pushed to code in a simple and straightforward fashion, and technically as advanced as possible, in order to unload the documentation burden to an appropriate level.  
	\item [Third] From the business perspective, Agile development is meant to produce business value right after the project starts so that the develop team can minimize the risks that the contract can not be fulfilled. Therefore the relationship and cooperation between the developers and the customers are highly recommended to be written strictly in the contracts, even though the rest of the contracts can still grow at the same pace at the size of the software project. In this way a feasible and productive relationship can be established and maintained for the two parties.
	\item [Fourth] In Agile development process, the development team consists of software developers and representatives for the customers. They work together in an intensive and iterate manner. Both of them should be well-informed, competent and authorized to bring in the proper adjustment on the new emerging needs during the development process life-cycle. This requires that any of the development participants should be used to the often changing work requirements and ready for making changes. In the mean time, the contracts should be formed in the way that can facility the modifications and enhancements on them later.       
\end{description}

Based on its key values, the core of Agile software development methods should be light but sufficient, and it has to obey the human- and communication-oriented rules. The Agile process is both light and sufficient.  Lightness means that it is easy to control and change, whereas sufficiency means that it can supply the complete plan for the development and solve all the possible problems popping up during the development. Therefore, Alistair Cockburn \citep{Cockburn2006} proposed the following factors to increase the possibilities to be a successful project:

\begin{itemize}
	\item Two to eight people in one room. This helps with communication and community between team members.
	\item Onsite usage experts. The experts are highly relied on the short and continuous feedback cycles.
	\item Short increments. Usually the time span of one to three months is quite practical for testing and repairing.
	\item Fully automated regression tests. Unit and functional tests stabilize code and allow continuous improvement. 
	\item	Experienced developers. The empirical statistics shows that experience accelerates the development time from 2 to 10 times in comparison to slower team members.
\end{itemize}

\section{Comparisons}

Nowadays in the software business the changing environment is inevitable, which directly influences the software development processes. Therefore, in the software development, the team members should not seek the way to stop changes from happening in the project; instead, they have to prepare the methods to handle the upcoming changes throughout the development life cycle. One thing needs to bear in mind is that it is more important to satisfy the customers at the time of delivery rather than make them happy at the beginning of the project. To handle the situation like that, the Agile development team should produce the first delivery in weeks, and achieve an early win and rapid feedbacks. After analysis of each of the feedbacks, the development team can continue to invent simple solutions so that there will be less to change in next time or the changes will be done easier. Besides, the development team should keep improving the design quality in order to make the next iteration less expensive to implement. And also, during the development, the development team should do the test on a constant basis, and try to detect the errors and mistakes as early as possible for the purpose of less fix cost.

In comparison to the traditional development, Agile has some quite different ideas. The general philosophy of Agile methods has quite high requirement on team members' quality, because Agile believes that only unforgiving and honest working atmosphere, loyal teamwork spirit and high-motivated people sharing the same goal can gain the maximal results. Beside this, Agile process believes that too much documentation is not necessary. As a matter of fact, Agile process promotes the birth of simple solutions, which don't need overweight documentation, either. Similarly, big up-front design is not required. The process itself consists of many iterative courses. Many expected changes will happen on the original design. To put too much effort for a big design will turn out to be a waste. Opposed to the reduction in paper work, the importance of communications raises itself in Agile process. Members must have frequent and regular discussion to make sure what they are doing is right. Table \ref{tab:compareagiletoothers} gives the comparison between Agile methods and other software development paradigms. 
%{| p{3cm} | p{3.5cm} | p{3.5cm} | p{4.4cm} |}
\begin{table}
	\centering
		\begin{tabular}{| p{0.20\textwidth} | p{0.22\textwidth} | p{0.22\textwidth} | p{0.20\textwidth} |}
		\hline
		\textbf{Home-ground area} & \textbf{Agile methods} & \textbf{Open-source software} & \textbf{Plan-driven methods} \\ [3ex]
		\hline
		Developers & Agile, knowledgeable, collocated, and collaborative & Geographically distributed, collaborative, knowledgeable and agile teams & Plan-oriented; adequate skills; access to external knowledge \\ [3ex]
		\hline
		Customers & Dedicated, knowledgeable, collocated, collaborative, representative, and empowered & Dedicated, knowledgeable, collaborative, and empowered & Access to knowledgeable, collaborative, representative, and empowered customers \\ [3ex]
		\hline
		Requirements & Largely emergent; rapid change & Largely emergent; rapid change, commonly owned, continually evolving never finalized & Knowable early; largely stable\\ [3ex]
		\hline
		Architecture & Designed for current requirements & Open, designed for current requirements & Designed for current and foreseeable requirements \\ [3ex]
		\hline
		Refactoring & Inexpensive & Inexpensive & Expensive \\ [1ex]
		\hline
		Size & Smaller teams and products & Larger dispersed teams and smaller products & Larger teams and products \\ [3ex]
		\hline
		Primary objective & Rapid value & Challenging problem & High assurance \\ [3ex]
		\hline	
		\end{tabular}
	\caption{Compare Agile with Other Development Paradigms (Source: \cite{Boehm2002})}
	\label{tab:compareagiletoothers}
\end{table}

\section{Characterization}

The core idea of Agile process is to condense the development life-cycle of projects into successive iterative courses, in order that it can realize the fast delivery purpose. From this perspective, the Agile process can be characterized by the following \citep{Miller2001}:
\begin{enumerate}
	\item Modularity on development process level
	\item Iterative with short cycles enabling fast verifications and corrections
	\item Time-bound with iteration cycles from one to six weeks
	\item Parsimony in development process removes all unnecessary activities
	\item Adaptive with possible emergent new risks
	\item Incremental process approach that allows functioning application building in small steps
	\item Convergent (and incremental) approach minimizes the risks
	\item People-oriented, i.e. Agile processes favor people over processes and technology
	\item Collaborative and communicative working style 
\end{enumerate}

\section{Scrum}

The word \textit{Scrum} is originally the strategic item in the game of rugby. It means \textit{getting an out\-of\-play ball back into the game} with teamwork. Later the engineers borrow and develop this approach for the systems development process. In the first place, the engineers wanted systematic method to bring in flexibility, adaptability and productivity into the industrial production. They took advantage of industrial process theory and adopted its ideas on the systems development. Under this background, Scrum was born from the daily practices. However, one thing that needs to be clarified is Scrum is not oriented for the software development techniques in the implementation phase. Instead, Scrum focuses on how the team members should work in a frequently changing environment so that they can still construct the system in the flexible fashion.  

Scrum notices that several environmental and technical variables, e.g. requirements, time frame, resources and technology, change during the systems development process all the time. These variables are the reason why the development processes are unpredictable and complex. Accordingly, flexibility is required for the systems development process so that the process can respond to the changes well. This helps greatly produce a useful system at the delivery. 

In Scrum process, frequent management activities are required to take place all the time. These management activities are responsible for keeping detecting any insufficiencies and mistakes happening in the development. Therefore, Scrum is great to enhance the existing engineering practices. 

\subsection{Scrum Roles and Responsibilities}

There are four identifiable Scrum roles: the \textit{Scrum Master}, the \textit{Product Owner}, the \textit{Team} and \textit{Customer}. The four roles have different tasks and purposes during the process and its practices. All management responsibilities in a project are divided among the four roles. The following definitions are presented based on the reference to \citep{Schwaber2004}.

\subsubsection{Scrum Master}

Scrum Master is a new management role introduced by Scrum. The Scrum Master is responsible for Scrum Process. He teaches Scrum to everyone involved in the project. He makes sure that everyone follows Scrum rules and practices. Scrum Master implements Scrum so that it fits within a company's culture and still produces the expected benefits. He needs to encourage the project team as well as the customer frequently to keep them working as productively as possible. 

\subsubsection{Product Owner}

Product Owner is selected by the Scrum Master. He is responsible for representing the interests of everyone with a stake in the project and its resulting system. He creates the project's initial overall requirements, return on investment objectives and release plans so that the initial and ongoing funding can be achieved. The list of requirements is called the Product Backlog. The Product Owner prioritizes the Product Backlog to queue up the most valuable requirements for the next iteration. In this way the most valuable functionality is produced first and built upon. Besides, he also participates in estimating the development effort for Product Backlog items and turns the issues in the Product Backlog into features to be developed. 

\subsubsection{Scrum Team}

Scrum Team is responsible for developing functionality. They have the authority to decide on the necessary actions and to organize itself in order to achieve the goals of each Sprint. The Scrum Team is self-managing, self-organizing, and cross-functional. They are the persons who need to figure out how to turn Product Backlog into an increment of functionality within an iteration and managing their own work to do so. They are involved in effort estimation, creating the Sprint Backlog, reviewing the Product Backlog list and suggesting impediments that need to be removed from the project. Therefore, each of the team members is responsible for the success of each interaction and of the project as a whole. 

\subsubsection{Customer}

Customer is always the inputs and feedbacks about the satisfaction of the current product and progress. He is just part of the tasks with respect to Product Backlog items for the system being developed or enhanced. 

\subsection{Scrum Practices}

A set of Scrum practices and rules establishes an environment within which products can be rapidly and incrementally built in complex environments. These practices have been established experientially through thousands of Scrum projects. During the evolution of Scrum in thousands of development projects, these practices change themselves. And also based on different situation and needs, different Scrum projects may employ different sets of Scrum practices. However, the common and most used practices remain. 

\subsubsection{Product Backlog}

Product Backlog is an evolving, prioritized queue of business and technical functionality that needs to be developed into a system. It defines everything that is needed in the final product based on current knowledge. Backlog items can include, for example, features, functions, bug fixed, defects, requested enhancements and technology upgrades. The Product Backlog is never complete, and the Product Backlog used in the project plan is only an initial estimate of the requirements. The Product Backlog gets updated as the product and environment where it will be used changes. The Product Backlog is dynamic. It is constantly changed to match what the product needs to be appropriate, completive and useful. As long as a product exists, the Product Backlog also exists. Figure \ref{fig:productbacklog} shows an example Product Backlog\footnote{\url{http://www.mountaingoatsoftware.com/product_backlog}} from a real project. 

\begin{figure}
	\centering
		\includegraphics[width=\textwidth]{../pic/productbacklog.jpg}
	\caption{A Product Backlog From A Real Project}
	\label{fig:productbacklog}
\end{figure}


\subsubsection{Sprint}

A Sprint is a fixed period of time when the team works for the project. It is the procedure of adapting to the changing environment variables, e.g. requirements, time, resources, knowledge and technology. The Scrum Team decides what it will accomplish during the upcoming Sprint, and then organizes itself to produce a new executable product increment in a Sprint that lasts approximately thirty calendar days. A Sprint has certain rules as the following \citep{Controlchaos.com2003}.

\begin{itemize}
	\item A sprint lasts no more than 30 days.
	\item A sprint is undertaken by a cross functional team consisting of no more than 9 members.
	\item Every sprint has a specific goal.
	\item An executable demonstrating the goal will be completed by the team during the sprint.
	\item The sprint team has final call in estimating and determining what they can accomplish during the sprint.
	\item Once the sprint is underway, new backlog cannot be added to the sprint except that, if the scrum master determines that a new backlog item will enhance the viability of the product, is in alignment with the sprint, builds on the sprint's executable, and can be completed within the sprint's time frame, the backlog item can be added. Examples are building a demonstration of the executable for a specific purpose, such as a trade show or prospect.
	\item If external forces determine that the sprint is working on the wrong thing, a sprint can be halted and restarted with new backlog and purpose.
\end{itemize}

The tools that the team use are Sprint Planning Meetings, Sprint Backlog and Daily Scrum Meetings. Figure \ref{fig:sprintinputs} shows a Sprint with its practices and inputs. 

\begin{figure}
	\centering
		\includegraphics[width=\textwidth]{../pic/sprint_inputs.jpg}
	\caption{Practices and Inputs of Sprint (Source: \cite{Pekka2002})}
	\label{fig:sprintinputs}
\end{figure}

\subsubsection{Sprint Planning Meeting}

A Sprint Planning Meeting is a two-phase meeting organized by the Scrum Master. It actually consists of two consecutive meetings. In the first meeting, the Scrum Team meets with the Product Owner, the customers and the management to work out a plan of what functionality to build during the next Sprint. In the second meeting the Scrum Team figures out by themselves how it is going to build this functionality into a product increment during the next Sprint. Input to the meeting is the Product Backlog, the latest increment of product, and the capabilities and past performance of the Scrum Team. Figure \ref{fig:input_for_new_sprint} shows an overview of planning a new Sprint.

\begin{figure}
	\centering
		\includegraphics[width=\textwidth]{../pic/input_for_new_sprint.jpg}
	\caption{Input for A New Sprint}
	\label{fig:input_for_new_sprint}
\end{figure}

\subsubsection{Sprint Backlog}

The Sprint Backlog is a list of tasks that defines a Team's work for a Sprint. The list emerges during Sprint planning. The tasks on the Sprint Backlog are what the Scrum Team has defined as being required to turn committed Product Backlog items into system functionality. Each task identifies who is responsible for doing the work and the estimated amount of work remaining on the task on any given day during the Sprint.

The Sprint Backlog is very commonly maintained as an Excel spreadsheet but it is also possible to use your defect tracking system or any of a number of software products designed specifically for Scrum or Agile. Figure \ref{fig:sprintbacklog} shows a Sprint Backlog\footnote{\url{http://www.mountaingoatsoftware.com/sprint_backlog}} in Excel.

\begin{figure}
	\centering
		\includegraphics[width=0.75\textwidth]{../pic/sprintbacklog.jpg}
	\caption{A Sprint Backlog in Excel}
	\label{fig:sprintbacklog}
\end{figure}

\subsubsection{Daily Scrum Meeting}

Scrum method is a complex process that requires lots of communications. The Daily Scrum Meeting is where the team members can come to communicate. It is conducted by the Scrum Master. Each Scrum Team meets very day for a status meeting of 15 minutes. In these 15 minutes, the team members talk about what they have got so far, what they are supposed to finish before the next meeting, and what difficulties they will probably encounter. The Daily Scrum Meeting is not a status update meeting in which a boss is collecting information about who is behind schedule. Rather, it is a meeting in which team members make commitments to each other. If a programmer stands up and says \"Today I will finish the data storage module\" everyone knows that in tomorrow's meeting he will say whether or not he did finish.  Besides the Scrum Team also the management can join in the meeting. 

\subsubsection{Sprint Review Meeting}

At the end of each Sprint a Sprint Review Meeting is held. The Sprint Review Meeting is a four-hour informational meeting. During the meeting, the Scrum Team and the Scrum Master present what they have accomplished in the Sprint to the management, customers, and the Product Owner. The project is assessed against the Sprint goal determined during the Sprint Planning Meeting. The Sprint Review Meeting is not supposed to be bring in any distracting effect into the team, whereas just the natural result of the Sprint.

\subsection{Scrum Flow}

A vision of the system to be developed is the beginning of a Scrum project. The vision might be blurry in the first place, but it will become increasingly clearer as the project moves forward. The Product Owner needs to picture a very promising perspective to the customers funding the project. The customers should be convinced that the delivery of the product will maximize their Return On Investment (ROI).  The Product Owner makes up a Product Backlog which lists the functional and nonfunctional requirements. The Product Backlog will convey the picture to the customers when the requirements are realized into functionality. The Product Backlog prioritizes all the items which can produce the most profits at the top, and divides itself into proposed releases. However, the prioritized Product Backlog is just a starting point. As the business requirements changes and the Scrum Team develop the required features on the Product Backlog into functionality quickly or slowly, the Product Backlog changes all the time. As a matter of fact, such changes are supposed to happen, and it happens once the project starts. 

After the Product Backlog is formulated, the Scrum Team can start to work on the Sprint basis. Each Sprint is an iteration of 30 successive calendar days. Every Sprint starts with a Sprint Planning Meeting. In the meeting, the Product Owner chooses the items in the Product Backlog from the highest priority, and discus with the Scrum Team what should be done in the next Sprint. The Product Owner tells the team what is wanted and the Scrum Team tell the Product Owner that how much of what he wants can be developed within the next Sprint, according to their knowledge. The Sprint Planning Meeting should not last over 8 hours, because the too long back-and-forth discussion is not good for making up the mind. The goal of Scrum is to get to work, rather than to think about working. 

The Sprint Planning Meeting consists of two parts. Each of them is four hours. In the first hours, the Product Owner presents the Scrum Team what items have the highest priorities on the Product Backlog. The Scrum Team ask questions about the content, purpose, meaning and intentions of these items. Before the first four hour passes, the Scrum Team need to pick up as many items in the Product Backlog as they think they can turn them into product functionalities and deliver them by the end of the next Sprint. The Scrum Team has to commit to the Prodcut Owner that they will do their best. In the second four-hour Sprint Planning Meeting, the Scrum Team makes up the plan for the next Sprint. The Scrum Team is supposed to mange their work on their own, therefore, they have to figure out a provisional plan for the next Sprint. They formulate the Sprint Backlog and put the tasks in it. The tasks are gradually finished as the Sprint is progressing. At the start of the second four-hour Sprint Planning Meeting, the next Sprint has actually started, and it will come to the end in the following 30 days.

In the every morning of the Sprint, the Scrum Team need to gather up for a 15-minute meeting. This is the Daily Scrum Meeting. In the meeting, the team members usually answer three questions: What have you done on the project since the last Daily Scrum Meeting? What do you plan to do on this project between now and the next Daily Scrum Meeting? What are the difficulties and troubles can you see standing in the way of you realizing your commitments to this Sprint? This meeting intends to synchronize the team members' daily work, and motivate them working in the most efficient way. 

When the Sprint comes to the end, a Sprint Review Meeting is held. It usually lasts for four hours. In this time, the Scrum Team presents what they developed in the Sprint to the Product Owner and customers who want to attend. This informal meeting is supposed to gather people together to collectively decide what the Scrum Team should do in the next Sprint. After the Sprint Review Meeting and before the next Sprint Planning Meeting, the Scrum Master holds another three-hour meeting with the Scrum Team. In this meeting, the Scrum Master encourage the team to evaluate how they complied with the Scrum process and practices. The evaluation will help the team more effectively work for the next Sprint. Together, the Sprint Planning Meeting, the Daily Scrum Meeting and the Sprint Review Meeting make up each of the iterations of the Scrum process, and roll the development till the end. Figure \ref{fig:scrum_flow} shows the overview of the Scrum process. 

\begin{figure}
	\centering
		\includegraphics[width=1.00\textwidth]{../pic/scrum_flow.jpg}
	\caption{Scrum Flow Overview (Source: \cite{Mahnic2005})}
	\label{fig:scrum_flow}
\end{figure}
